Toward a Knowledge-Driven Context-Aware System for Surgical Assistance

Complex surgeries complications are increasing, thus making an efficient surgical assistance is a real need. In this work, an ontology-based context-aware system was developed for surgical training/assistance during Thoracentesis by using image processing and semantic technologies. We evaluated the Thoracentesis ontology and implemented a paradigmatic test scenario to check the efficacy of the system by recognizing contextual information, e.g. the presence of surgical instruments on the table. The framework was able to retrieve contextual information about current surgical activity along with information on the need or presence of a surgical instrument

Development of an intelligent surgical training system for Thoracentesis

Surgical training improves patient care, helps to reduce surgical risks, increases surgeon’s confidence, and thus enhances overall patient safety. Current surgical training systems are more focused on developing technical skills, e.g. dexterity, of the surgeons while lacking the aspects of context-awareness and intra-operative real-time guidance. Context-aware intelligent training systems interpret the current surgical situation and help surgeons to train on surgical tasks. As a prototypical scenario, we chose Thoracentesis procedure in this work. We designed the context-aware software framework using the surgical process model encompassing ontology and production rules, based on the procedure descriptions obtained through textbooks and interviews, and ontology-based and marker-based object recognition, where the system tracked and recognised surgical instruments and materials in surgeon’s hands and recognised surgical instruments on the surgical stand. The ontology was validated using annotated surgical videos, where the system identified “Anaesthesia” and “Aspiration” phase with 100% relative frequency and “Penetration” phase with 65% relative frequency. The system tracked surgical swab and 50 mL syringe with approximately 88.23% and 100% accuracy in surgeon’s hands and recognised surgical instruments with approximately 90% accuracy on the surgical stand. Surgical workflow training with the proposed system showed equivalent results as the traditional mentor-based training regime, thus this work is a step forward a new tool for context awareness and decision-making during surgical training

A knowledge-based framework for task automation in surgery

Robotic surgery has significantly improved the quality of surgical procedures. In the past, researches have been focused on automating simple surgical actions, however there exists no scalable framework for automation in surgery. In this paper, we present a knowledge-based modular framework for the automation of articulated surgical tasks, for example, with multiple coordinated actions. The framework is consisted of ontology, providing entities for surgical automation and rules for task planning, and \u201cdynamic movement primitives\u201d as adaptive motion planner as to replicate the dexterity of surgeons. To validate our framework, we chose a paradigmatic scenario of a peg-and-ring task, a standard training exercise for novice surgeons which presents many challenges of real surgery, e.g. grasping and transferring. Experiments show the validity of the framework and its adaptability to faulty events. The modular architecture is expected to generalize to different tasks and platforms

Approaches for action sequence representation in robotics: a review

Robust representation of actions and its sequences for complex robotic tasks would transform robot’s understand- ing to execute robotic tasks efficiently. The challenge is to under- stand action sequences for highly unstructured environments and to represent and construct action and action sequences. In this manuscript, we present a review of literature dealing with representation of action and action sequences for robot task planning and execution. The methodological review was conducted using Google Scholar and IEEE Xplore, searching the specific keywords. This manuscript gives an overview of current approaches for representing action sequences in robotics. We propose a classification of different methodologies used for action sequences representation and describe the most important aspects of the reviewed publications. This review allows the reader to understand several options that do exist in the research community, to represent and deploy such action representations in real robots

Approaches for action sequence representation in robotics: a review

Robust representation of actions and its sequences for complex robotic tasks would transform robot’s understand- ing to execute robotic tasks efficiently. The challenge is to under- stand action sequences for highly unstructured environments and to represent and construct action and action sequences. In this manuscript, we present a review of literature dealing with representation of action and action sequences for robot task planning and execution. The methodological review was conducted using Google Scholar and IEEE Xplore, searching the specific keywords. This manuscript gives an overview of current approaches for representing action sequences in robotics. We propose a classification of different methodologies used for action sequences representation and describe the most important aspects of the reviewed publications. This review allows the reader to understand several options that do exist in the research community, to represent and deploy such action representations in real robots

Towards a robot task ontology standard

Ontologies serve robotics in many ways, particularly in de- scribing and driving autonomous functions. These functions are built around robot tasks. In this paper, we introduce the IEEE Robot Task Representation Study Group, including its work plan, initial development efforts, and proposed use cases. This effort aims to develop a standard that provides a comprehensive on- tology encompassing robot task structures and reasoning across robotic domains, addressing both the relationships between tasks and platforms and the relationships between tasks and users. Its goal is to develop a knowledge representation that addresses task structure, with decomposition into subclasses, categories, and/or relations. It includes attributes, both common across tasks and specific to particular tasks and task types

Enriching surgical process models by BPMN extensions for temporal durations

Many surgical interventions are finding new techniques in robot-assisted surgery, which allows surgeons to perform surgery with the help of robotic arms. A formal representation of robot-assisted surgery can provide surgeons with an overview of the main stages of surgical intervention and a detailed description of the different steps, including all the possible emergencies that may occur. Formalizing such kinds of interventions could also help to train new surgeons. However, literature does not consider formal representations and properties of robot-assisted surgery properly. The Business Process Model and Notation (BPMN) is a standard language allowing to represent processes in a graphical and semi-formal way. In this paper, we propose to use BPMN for representing the processes and the guidelines underlying robot-assisted surgery, considering the explicit modeling of temporal and informational aspects: in detail, guidelines aim at providing surgeons with high-level recommendations based on the operational knowledge of expert users, delivering hints on how to execute exploration tasks. As a real-world application domain, we consider here the Robot-Assisted Partial Nephrectomy (RAPN), which is the partial surgical removal of a kidney to treat severe kidney diseases such as cancer